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Abstract 


This report presents a review of literature on approaches to adaptive and adaptable task/function 
allocation and adaptive interface technologies for effective human management of complex systems that 
are likely to be issues for the Next Generation Air Transportation System, and a focus of research under 
the Aviation Safety Program, Integrated Intelligent Flight Deck Project. Contemporary literature retrieved 
from an online database search is summarized and integrated. The major topics include the effects of 
delegation-type, adaptable automation on human performance, workload and situation awareness, the 
effectiveness of various automation invocation philosophies and strategies to function allocation in 
adaptive systems, and the role of user modeling in adaptive interface design and the performance 
implications of adaptive interface technology. 

The main finding of the review is that adaptable system designs requiring human delegation of task and 
function authority to automation during performance may pose an additional workload on operators 
leading to degradations in situation awareness and performance, as compared to adaptive automation. In 
adaptively automated systems, dynamic task/function allocations are typically managed by a computer 
based on real-time monitoring of operator workload states and decision making regarding the allocation 
of system control is not an additional responsibility for operators. It is also revealed that approaches to 
adaptable system design may only be effective for addressing finite operating states of systems and that 
such automation may only produce performance comparable to manual control under variable 
environmental conditions. Beyond this, the findings on adaptive interfaces for implementing adaptive 
automation include the need for consideration of multiple user characteristics, preferences and behaviors 
in the design process. It is also observed that adaptive interface feature/components must be linked to 
specific task requirements to promote performance and that it is necessary to maintain some consistency 
in interface design across modes of system operation to allow operators to effectively apply mental 
models. 

In general, the findings of this review motivate further exploration of adaptively automated systems for 
managing complex system operator workload states and attempting to promote situation awareness, as 
compared to design of adaptable system alternatives. This includes: (1) exploration of types and levels of 
adaptive automation that are easiest for operators to work with in different task contexts, (2) development 
of more integrated and robust approaches to operator state monitoring, (3) development of highly accurate 
methods for operator state classification and computer triggering of adaptive task/function allocations, 
and (4) development of on-line methods for assessing adaptive automation effectiveness for operator 
workload management and maintenance of situation awareness. 



1. Introduction 


1.1. Defining Automation 

Automation refers to "...systems or methods in which many of the processes of production are 
automatically performed or controlled by autonomous machines or electronic devices" (Parsons, 1985). 
Automation is a tool, or resource, that the human operator can use to help perform some task that would 
be difficult or impossible without machine aiding (Billings, 1997). Therefore, automation can be thought 
of as a process of substituting the activity of some device or machine for some human activity, fully or 
partially; or it can be thought of as a state of technological development. Automation does not have to be 
an “all or none” option, but can take forms varying in the type and level of aiding provided to operators, 
as shown in Tables 1 and 2. 

Table 1. Sheridan’s (1987) Levels of Human- Automation Interaction. 

1. The computer offers no assistance; the human must do it all 

2. The computer offers a complete set of action alternatives 

3. The computer narrows the selection down to a few 

4. The computer suggests a selection, and 

5. Executes that suggestion if the human approves, or 

6. Allows the human a restricted time to veto before automatic execution, or 

7. Executes automatically, then necessarily informs the human, or 

8. Informs the human after execution only if he asks, or 

9. Informs the human after execution if it, the computer, decides to 

10. The computer decides everything and acts autonomously, ignoring the human 

Table 2. Endsley and Kaber’s (1999) Levels of Automation Taxonomy. 


ROLES 


LEVEL OF 
AUTOMATION 

MONITORING 

GENERATING 

SELECTING 

IMPLEMENTING 

1 . Manual Control 

Human 

Human 

Human 

Human 

2. Action Support 

Human/ Computer 

Human 

Human 

Human/ Computer 

3. Batch 
Processing 

Human/Computer 

Human 

Human 

Computer 

4. Shared Control 

Human/ Computer 

Human/Computer 

Human 

Human/Computer 

5. Decision 
Support 

Human/Computer 

Human/Computer 

Human 

Computer 

6. Blended 
Decision 
Making 

Human/Computer 

Human/Computer 

Human/Computer 

Computer 

7. Rigid System 

Human/Computer 

Computer 

Human 

Computer 

8. Automated 
Decision 
Making 

Human/Computer 

Human/Computer 

Computer 

Computer 

9. Supervisory 
Control 

Human/Computer 

Computer 

Computer 

Computer 

10. Full 
Automation 

Computer 

Computer 

Computer 

Computer 
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Many researchers have questioned whether automation should be viewed as a substitution of one agent 
for another; that is, whether machines designed by humans can or should be the ideal replacement for 
humans. This is the idea of “apparent simplicity, real complexity” advocated by some involved in human- 
automation research (e.g., Woods, 1996). Nevertheless, because many tasks performed by humans can be 
decomposed superficially into a myriad of task components, humans have sought to define what the 
human and machine can respectively do best. Such an approach to function allocation dates back to the 
“Fitts’ List” concept of the 1950s. However, as technology has advanced, the capability of machines to 
emulate human functions, sometimes inappropriately, has enabled engineers and systems designers to 
effectively automate most, if not potentially all, human functions. 

Today, the presence of automation has pervaded almost every aspect of modem lives. From the wheel to 
the modem jet aircraft, humans have sought to improve the quality of life. We have built machines and 
systems that not only make work easier, more efficient, and safe, but also give us more leisure time. The 
advent of automation has further enabled us to achieve this end. With automation, machines can now 
perform many of the activities that we once had to do. Our automobile transmission will shift gears for 
us. Our airplanes will fly themselves. All we have to do is turn the machine “on” and “off’. It has been 
suggested that one day there may not even be a need for humans to do so “through” the proverbial switch. 

1.2. The “New” Problems of Automation 

"During the 1970s and early 1980s. ..the concept of automating as much as possible was 
considered appropriate. The expected benefit was a reduction in pilot workload and 
increased safety... Although many of these benefits have been realized, serious questions 
have arisen and incidents/accidents that have occurred which question the underlying 
assumptions that a maximum available automation is always appropriate or that we 
understand how to design automated systems so that they are fully compatible with the 
capabilities and limitations of the humans in the system." 

— Air Transport Association of America (AT A), 1989 


The real question is not “...whether one or another function can be automated, but, rather, whether it 
should be” (Wiener & Curry, 1980, p.2). “Although aviation has become a remarkably safe way to move 
people and goods, preventable accidents continue to occur. To an increasing extent, these accidents 
involve both human operators and their machines, because the humans and machines have become more 
interdependent.... Automation is now a central element in that system. It has been extremely successful 
in improving the reliability and productivity of the system. Like all technology, its successes have 
brought with them new problems to solve” (Billings, 1997, p. 297). 

Automation is "...a wrapped package — a package that consists of many different dimensions bundled 
together as a hardware/software system. When new automated systems are introduced into a field of 
practice, change is precipitated along multiple dimensions" (Woods, 1996). As a consequence, 
automation increases the burdens and complexities for those responsible for operating, troubleshooting, 
and managing systems resulting in “new” problems, including: 

1 . adding to or changing the nature of the task, such as device setup and initialization, configuration 
control, and operating sequences; 

2. changing and/or adding cognitive demands; 

3. changing the roles of people in the system, often relegating people to supervisory controllers who 
watch over the automation potentially leading to “out-of-the-loop” performance issues; 
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4. increasing coupling and integration among parts of a system often resulting in data overload and 
lack of "transparency” leaving operators to wonder “what is it [the automation] doing now?”; and 

5. lack of appreciation of the impact of automation on humans by those who advocate for the use of 
the technology. 

These issues are not singular to aviation alone. Research has shown that these new problems have had 
significant effects on human operators, who work in a variety of high technology environments, which 
include effects of reduced job satisfaction (e.g., automation may dehumanize human roles), lowered 
vigilance, fault-intolerant systems, silent failures, an increase in cognitive workload, automation-induced 
failures, over-reliance, complacency, decreased trust, manual skill erosion, false alarms, and a decrease in 
mode awareness (e.g., Parasurman, Molloy & Singh, 1993a; Shiff, 1983; Sarter & Woods, 1995). These 
are in addition to numerous fatal accidents, directly attributable to poor human-automation interaction, 
which serve as testimony to the need for new approaches to human-automation design. Two very 
promising approaches are adaptable and adaptive automation systems. 

1.3. Overview of Memorandum 

This report presents a review of literature on approaches to adaptive and adaptable function allocation, 
and adaptive interface technologies, for effective human management of complex systems that are likely 
to be issues for the Next Generation Air Transportation System, and a focus of research under the 
Aviation Safety Program, Integrated Intelligent Flight Deck Project.. Adaptive automation has been 
defined as the dynamic allocation of complex system control to a human or computer controller, based on 
system-state variables, with the objective of optimizing performance, reducing operator workload and 
promoting situation awareness (Rouse, 1977; Parasuraman, 1987; Scerbo, 1996; Kaber & Riley, 1999). In 
adaptive systems, the overall degree of autonomy or the number of functions operating under automation 
can be modified in real time (Scerbo, 1996) in order to best address workload demands. In adaptive 
systems, authority over function allocations typically remains with the computer, which mandates the role 
of the operator. However, it is possible that the human operator and a computer may share authority in 
invoking automation of specific functions. Adaptable systems differ from adaptive systems in that the 
automation invocation authority remains with the human operator. The human is responsible for initiating 
changes between modes of automation and may also have control over system interface settings (Bailey, 
Scerbo, Freeman, Mikulka & Scott, in press). Prior research (Scerbo, 1996) has suggested that there may 
be performance differences among adaptive and adaptable systems and some recent work has made direct 
comparison of human functioning with adaptive versus adaptable systems (Bailey et al., in press). 

The objective of this review was to detail the state of research on the design and implementation of 
adaptive and adaptable automation systems. This included identifying effective approaches for promoting 
operator performance with complex systems and managing workload, making general recommendations 
on adaptive automation and adaptive interface design, and projecting future research needs in the area. A 
literature search was conducted on research on human performance with adaptable systems, adaptive task 
and function allocation in automated systems, and the design of adaptive interface technologies for 
delivering adaptive automation. The search involved use of an on-line reference database, and reviews of 
references used in prior technical reports on adaptive automation research were also completed. The 
online research database includes all articles published in registered archival journals dating back to 1945. 
The review summarizes the advantages and disadvantages of adaptable and adaptive automation for 
complex systems control and identifies benefits and limitations of approaches to adaptive interface design 
for providing users with access to adaptive automation. 
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2. Adaptable Systems 


Scerbo (1996) formally defined adaptable automation and empirical work on truly adaptable systems 
(versus adaptive automation involving some human invocation authority over dynamic function 
allocations) has only recently appeared in the literature (e.g., Bailey et al., in press; Miller, 2003; 
Parasuraman, Galster, Squire, Furukawa & Miller, 2005; Squire, Trafton & Parasuraman, 2006). These 
studies have yielded mixed results on the potential utility of adaptable systems for promoting performance 
and managing operator workload, as compared to adaptive automation, high-level static automation and 
manual control. 

Bailey et al. (in press) investigated performance, situation awareness and workload effects of an adaptive 
system in which dynamic function allocations were based on operator workload states measured using an 
EEG-based index of engagement with an adaptable system in which control allocations were invoked by 
operators without suggestions from a computer. They used a modified version of the multi-attribute 
(flight) task battery (MAT-B) that included additional simulated aircraft gauges, which served as a basis 
for situation awareness assessment. In one experiment, the authors found no significant effect of the 
adaptive automation condition on task performance, as compared to a condition in which dynamic 
function allocations occurred at random. However, in a second experiment, they found the adaptable 
system condition (i.e., subjects could invoke automation or manual control, at will) to worse performance 
than the arbitrary automation condition in resource management task performance. They also found a 
significant reduction in workload for subjects exposed to arbitrary automation versus adaptable 
automation. When subjects were not responsible for managing dynamic control allocations, their ratings 
of subjective workload were lower. Unfortunately, there was no comparison of the adaptive or adaptable 
system results with completely manual or fully automated control conditions. However, Bailey et al. did 
compare the results of their first and second experiments and they found that the adaptive condition 
yielded lower subjective ratings or workload than the adaptable system. The authors attributed this to the 
requirement for subjects to manage control allocations as part of the adaptable condition. 

Miller and Parasuraman (2003) presented a new approach to flexible automation for complex systems to 
extend beyond the historical taxonomies of levels of automation (e.g., Sheridan and Verplank, 1987) and 
use of stages of human information processing as a basis for defining types and levels of automation 
(Parasuraman, Sheridan & Wickens, 2000). They said that automation could be applied to every subtask 
to be addressed by a system as part of larger parent tasks. They argued for fine-grained models of tasks 
that could be shared between human operators and automation in adaptable systems. They introduced an 
approach to adaptable automation called the “playbook” approach involving operator selection of 
predefined automation behaviors to respond to changing environmental and task demands. The human 
operator is required to delegate tasks to the automation. Miller and Parasuraman (2003) presented an 
example application of the playbook approach to unmanned aerial vehicle (UAV) mission planning to 
illustrate how specific control subtasks could utilize different combinations of human and automation 
control. However, the study did not present any empirical results. 

On the basis of the research by Miller and Parasuraman (2003), Parasuraman, Galster, Squire, Furukawa 
& Miller (2005) conducted an empirical study of adaptable automation (involving use of the playbook 
approach) for human supervision of multiple robots in team performance of a flag capture task against an 
opponent robot team. In three experiments, they made comparison of human use of an interface for 
delegating tasks to the robots with manual control or with a combination of delegation automation and 
manual control. Other independent variables manipulated in the study included the number of robots 
being controlled and the postures of the opposing robot team (defensive, offensive or variable). In the first 
experiment, the playbook approach to automation allowed for some response to variable opponent robot 
behavior, but performance was never as good as when adaptable automation was applied to fixed 
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opponent behaviors (defense, offense). Beyond this, contrary to the authors’ hypothesis, there was no 
evidence that the playbook approach led to differences in perceived workload when subjects were posed 
with variable or fixed opponent behavior. Interestingly, the authors noted that operators tended to revert 
to manual control under high workload circumstances in order to deal with variable opponent robot 
behavior versus relying on automation to implement predefined robot behaviors. In their second 
experiment, Parasuraman et al. (2005) made comparison of completely manual control, fixed automation 
and flexible automation (the playbook approach). Surprisingly, the mode of control did not affect robot 
mission success rate and the flexible automation approach was no different from manual control in terms 
of task time. The fixed automation produced the worse performance, suggesting a “clumsy” 
implementation (Riley & Parasuraman, 1997). Most importantly, and related to the results of Bailey et al. 
(in press), the flexible automation produced the highest workload ratings and situation awareness was 
better for the manual control condition. The authors attributed decrements in operator situation awareness 
when using the playbook automation to increased workload. In their third experiment, Parasuraman et al. 
manipulated the degree of task automation available to operators and the number of robots being 
controlled. The forms of automation ranged from manual mode, to playbook automation, to a playbook 
approach with high-level behaviors (e.g., sending robots on the offensive). They found that use of 
automated control with fewer robots led to workload reductions compared to manual control. However, 
there was no affect of the automation on operator situation awareness compared to manual control. In 
general, manual control of multiple robots appeared to be no worse than the flexible automation in terms 
of workload and performance. Even though there was no conclusive evidence of workload reductions 
attributable to the flexible automation, the authors concluded by saying that automation was needed for 
high workload conditions in multiple robot control, but for novel task circumstances, manual control may 
be necessary. 

In another study, Squire, Parasuraman and Trafton (2006) used the same task, the same forms of 
automation and automation interfaces, as Parasuraman et al. (2005), in order to assess the performance 
costs of operator task switching in the robot flag capture. They made comparison of performance 
involving manual control of multiple robots with adaptable automation, specifically the playbook 
approach, involving computer control of low- or high-level robot functions. They found that manual 
control generally produced shorter task times than static automation, again suggesting a “clumsy” 
implementation of the automation (cf., Riley & Parasuraman, 1997), and the flexible/adaptable 
automation was no better than manual control. One advantage of the flexible automation they observed 
was that there were significantly fewer interface actions required of operators when they performed the 
task following a single strategy (e.g., offensive robot behavior), as compared to manual control and static 
automation. However, when operators were required to switch strategies/tasks during the robot flag 
capture, manual control produced the shortest task times followed by adaptable automation and then static 
automation. In general, the adaptable automation appeared to be no better than manual performance and 
only superior to “clumsy” static automation. Unfortunately, the authors did not record workload ratings, 
as a basis for further comparison of the adaptable (delegation-type) automation with manual control and 
the work of Parasuraman et al. (2005). 

2.1. Summary on Adaptable Systems Research 

On the basis of Bailey et al. (in press) research, it appears that adaptable systems may actually produce 
worse performance results than arbitrary automation of tasks in real-time. Adaptable automation may also 
lead to higher workload than adaptive automation, which yields performance no worse than arbitrary 
automation and better that adaptable automation. These are not promising observations for use of 
adaptable automation in complex task performance, like aircraft piloting, air traffic control, or human 
control of multiple robotic systems. With respect to the latter application, Parasuraman et al. (2005) work 
generally showed that adaptable systems, using a playbook-type approach to automation, appear to be 
suited to rule-based task performance under which task circumstances are known and predefined 
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behaviors can be applied. When task circumstances are dynamic and workload is high, operators tend to 
revert to the use of manual control (versus adaptable automation) in exhibiting knowledge-based 
behaviors. Related to Bailey et al. (in press) findings, Parasuraman et al (2005) found that adaptable 
automation performance may not be better than manual control and use of the playbook approach may 
lead to higher workload and worse situation awareness than manual control. (The authors’ interpretations 
of results were more liberal than this.) In addition, in one experiment by Parasuraman et al (2005), human 
manual control of multiple complex systems (simultaneously) appeared to be no worse than adaptable 
automation in terms of workload and performance, and it actually improved operator SA. This study 
provides evidence that human delegation of task automation (adaptable automation) may impose 
additional workload on operators potentially degrading SA; however, this was not one of the conclusions 
of Parasuraman et al. (2005). Finally, with respect to Squire et al. (2006) research, adaptable automation 
does not appear to improve performance over manual control, but it may reduce the number of interface 
actions for operators. This may translate into workload reductions, but no evidence was provided of such 
an effect. When operators are required to use multiple strategies in controlling complex systems and to 
switch between tasks, adaptable automation does not appear to provide a significant benefit over manual 
control, but it may offer some advantage relative to static, technology-centered automation. 

3. Adaptive Systems 


Human factors research on adaptive automation began in the early 1 990s with a number of seminal works 
by Parasuraman and colleagues (e.g., Parasuraman, 1993; Parasuraman, Bahri, Deaton, Morrison & 
Barnes, 1992; Parasuraman, Mouloua, Molloy & Hilbum, 1993). The first studies to use the terminology 
adaptive task allocation and adaptive function allocation appeared shortly after this (e.g., Rencken & 
Durrant-Whyte, 1993; Scallen, Hancock & Duley, 1995). In complex systems, high-level functions may 
include, for example, information analysis and decision making. Such functions may encompass multiple 
tasks to be addressed by an operator in system control (e.g., monitoring subsystem states, tracking a 
defined navigation path with a vehicle). Opposite to this, complex system tasks may involve multiple 
low-level functions. For example, a tracking task may involve the functions of compensatory tracking 
along a vertical axis or a horizontal axis. Research on adaptive automation has addressed both high- and 
low-level function allocation as well as adaptive task allocation. 

Many different approaches to adaptive task and function allocation have been developed. Early work 
investigated performance and workload implications of switching “on” or “off’ automation of specific 
types of tasks based on predefined schedules or models of human and computer control in multiple task 
scenarios (Parasuraman, 1993). Similar studies assessed performance-based approaches to adaptive 
automation in which computer assistance was provided to operators in, for example, monitoring tasks if 
their performance fell below a specified criterion (e.g., Parasuraman, Mouloua and Molloy, 1996). Other 
work has investigated workload-based approaches in which automation of tasks or functions occurs on the 
basis of operator workload states measured using secondary-task performance (Kaber & Riley, 1 999) or 
physiological indicators of cognitive load and arousal (e.g., Prinzel, Freeman, Scerbo, Mikulka & Pope, 
2000 (electro-encephalogram signals (EEG)); Wilson & Russell, 2003 (electro-cardiogram signals (ECG), 
electro-oculography (EOG), and respiration intervals)). Beyond this, other work has suggested that 
adaptive function allocation should occur on the basis of critical system states requiring human or 
automated handling of specific functions (see Scerbo (1996) for a thorough review of approaches). 

Adaptive automation can be applied to many different types of tasks and functions in complex systems in 
order to moderate operator workload yet maintain sufficient levels of task engagement for achieving 
situation awareness. Historical research has used the MAT-B to assess the affects of adaptive automation 
on human monitoring and tracking task performance (e.g., Parasuraman et al., 1996; Hilbum et al., 1993), 
as compared to completely manual or fully automated performance. This research concentrated on 
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potential benefits of adaptive automation for psychomotor task/function performance. Only in more 
recent work has the potential impact of adaptive automation on cognitive task workload and performance 
been assessed. Hilbum, Joma, Byrne and Parasuraman (1997) conducted a study on adaptive automation 
in the context of air traffic control. They examined whether decision-aiding automation could be used to 
reduce operator workload and optimize overall system performance. Experienced air traffic controllers 
were required to perform an airport arrival traffic simulation with or without the assistance of the 
automation. An automated Descent Advisor tool was simulated, which calculated aircraft trajectories, 
dynamically developed flight plans, detected planning conflicts, or projected separation conflicts. The 
tool offered the human operator advice to resolve conflicts. Hilbum et al. (1997) used three automation 
schemes including constant manual control, constant automation and the adaptive automation condition 
(under which the automation was invoked only during high traffic to simulate workload relief). They 
found that the adaptive automation condition resulted in the smallest increase in mental workload across 
trials. This research provided support for the use of automation and/or adaptive automation in cognitive 
(decision-making) tasks. However, the authors did not make comparison of adaptive automation of 
psychomotor tasks as part of air traffic control with adaptive automation of decision functions in order to 
determine the relative effectiveness of automating various information processing functions. 

Even more recently, Kaber, Wright, Prinzel and Clamann (2005) conducted a study to determine whether 
there are differential performance and workload effects of adaptive automation applied to various aspects 
of human information processing in a simulated air traffic control task. They investigated types of 
automation similar to those defined by Parasuraman et al. (2000), including information acquisition, 
information analysis, decision making and action implementation, adaptively applied to tasks performed 
in terminal radar approach control. Kaber et al. (2005) implemented adaptive automation using a 
workload-based approach in which automation of a single aspect of information processing occurred 
when operator performance in a secondary task fell below a specified criterion (i.e., the air traffic control 
workload increased substantially). They found that humans are generally better able to adapt to adaptive 
automation when applied to lower-order sensory and psychomotor functions, such as information 
acquisition and action implementation, as compared to adaptive automation applied to cognitive (planning 
and decision making) tasks. Like the prior investigations of adaptive automation of psychomotor 
functions, they also observed that any adaptive automation of information processing produced better 
performance than no automation whatsoever. With respect to operator workload, reductions also primarily 
occurred when adaptive automation was applied to psychomotor functions, including information 
acquisition and action implementation, in the air traffic control task. Adaptive automation of information 
analysis and decision making functions appeared to not only degrade performance under automated 
periods but it increased objective workload (measured with a secondary task) as well. 

The findings of the research by Hilbum et al. (1997) and Kaber et al. (2006) demonstrate benefits of 
adaptive automation, including model- and workload-based approaches, for supporting human 
performance of psychomotor and cognitive functions in complex systems control compared to manual 
control and static automation. There is also substantial evidence demonstrating workload reductions 
attributable to adaptive automation, as compared to manual control and static automation. These findings 
differ substantially from the contemporary results on adaptable systems. 

3.1. Automation Invocation Authority in Adaptive Systems 

As mentioned earlier, adaptively automated systems typically assign automation invocation authority to 
computers; however, some experimental setups have investigated forms of shared authority involving 
human and computer management of function allocations. Investigations of automation invocation 
authority in adaptive systems also began around the early 1990s (e.g., Hilbum, Molloy, Wong & 
Parasuraman, 1993) and several studies have followed (e.g., Scerbo, 1996; Bubb-Lewis & Scerbo, 1997; 
Clamann & Kaber, 2003). Hilbum, Molloy, Wong and Parasuraman (1993) evaluated human versus 
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computer managed allocation of functions in the context of the MAT-B with the possibility of applying 
automation to the monitoring and tracking tasks of the simulation in real-time. They found that 
performance in the tracking subtask of the MAT-B was significantly better when the computer 
determined manual and automated control allocations. In general, adaptive automation was found to 
produce better monitoring performance than purely manual monitoring. Hilbum et al. (1993) stated that 
performance degradations under adaptive automation observed during the experiment were due to 
operators frequently cycling between full automation and manual control (at will) and that excessively 
short automation cycles (2 min or less) compounded this effect. These results are in-line with the 
contemporary findings on adaptable automation reviewed above. When humans are required to manage 
function allocations at the same time they perform mission-related tasks, this may lead to workload 
increases and performance degradations. 

Kaber and Riley (1999) used a workload-based approach to adaptive automation to investigate the affects 
of computer mandated versus human elected function allocations on performance and workload in a 
radar-monitoring task. They used observations on secondary task performance (simple gauge monitoring) 
as a basis for mandating or suggesting to operators control allocations in aspects of the radar monitoring. 
Adaptive automation involving shifts between manual control and partial automation (a shared decision 
making mode) of the primary task were mandated for one group of operators and merely suggested for 
another. Kaber & Riley (1999) found significantly improved manual, primary task performance and 
enhanced secondary-task monitoring under automation for the mandated-adaptive automation group, with 
the opposite results occurring for non-mandated adaptive automation subjects. The average subject 
workload marginally exceeded an objectively established criterion by using the secondary task measure to 
direct adaptive automation of the primary task. This work demonstrated that adaptive automation could be 
used to effectively manage operator primary task workload within a defined range. It also provides 
additional evidence that computer decisions of whether and when automation should be invoked for 
certain system functions produces superior performance to requiring human operators to perform function 
delegation. This is consistent with other historical research (Scerbo, 1996) demonstrating that humans 
might not be the best judges of dynamic function allocations. 

In a more recent investigation, Clamann and Kaber (2003) assessed the performance and workload effects 
of applying adaptive automation to different stages of human-machine system information processing, 
including information acquisition, information analysis, decision making and action implementation, and 
facilitating dynamic function allocations through two levels of computer authority. The research was to 
provide insight into any interaction between these aspects of adaptive automation design. They 
hypothesized that adaptive automation of the higher-order information processing functions, including 
information analysis and decision making, would be more compatible with computer mandated 
allocations, while adaptive automation of lower-order functions, such as information acquisition and 
action implementation, would be more effective with partial human control (computer suggestion and 
human veto) of function allocations. Clamann and Kaber’s (2003) results demonstrated the effectiveness 
of adaptive automation to be dependent upon both the type of automation and the type of invocation 
authority designed into the system. Like the findings of Kaber et al. (2006), performance with adaptive 
automation of information acquisition was superior to performance under decision automation. However, 
contrary to the results of Kaber and Riley (1999), when using automated assistance, human invocation 
authority produced superior performance to computer mandates. The authors noted that subjects tended to 
remain under the automated control mode, which provided some advantage over manual control. Related 
to this, Clamann and Kaber (2003) observed that such control behavior might ultimately undermine any 
benefit of adaptive automation for operator situation awareness over the long run. In general, the results 
of this study suggested that a “blended” form of authority, involving humans invoking automation and 
computer mandates of returns to manual control as part of adaptive automation, may improve the 
potential for performance and workload benefits in complex systems. 



The findings of the studies on automation invocation authority in adaptive systems generally support the 
use of computer mandated function allocation over human delegation of tasks to a computer. Like the 
adaptable systems research, the main observation is that human responsibility for decisions on dynamic 
function allocations may contribute to workload and cause task performance problems. There is some 
data supporting the implementation of shared human and computer authority over function allocations 
depending upon the current mode of system control, but such findings may be highly system dependent. 
Beyond this, the scenario of a human operator invoking automation, only for a computer to subsequently 
mandate manual control may not be a form of blended invocation authority acceptable to operators in 
real-world systems. 

3.2. Adaptive Task Allocation Research 

The literature search revealed only six studies in the online research search database using the 
terminology adaptive task allocation to describe approaches to the design and implementation of adaptive 
automation in complex systems. Early work in this area studied the use of model- and performance-based 
approaches to adaptive task allocation and focused on supporting psychomotor task performance. In the 
majority of studies, entire system subtasks are allocated to either a human operator or automation and the 
degree of task responsibility is not defined at a lower, functional level. Parasuraman et al. (1996) 
examined the effect of such approaches to adaptive task allocation on human monitoring performance 
using the MAT-B during long duration tests. They required manual performance of the tracking and fuel 
management tasks and developed an automated engine status task. Under the adaptive automation 
strategies, manual control of the engine status task was periodically allocated based on when monitoring 
performance was expected to be at its worst (model-based adaptive automation) or if individual 
monitoring performance in previous automated periods did not meet criterion levels (performance-based 
adaptive automation). Automation failure detection was compared across the adaptive automation 
conditions and a static automation condition. In general, their results revealed that both adaptive 
automation approaches enhanced monitoring performance over static automation during long duration 
tests. 

Even prior to the work by Parasuraman et al. (1996), Rencken and Durrant- Whyte (1993) developed an 
approach to adaptive task allocation in a complex system involving the use of an adaptive human- 
computer interface for a security area surveillance task. They developed a quantitative model of human 
task performance in identifying and classifying targets to facilitate adaptive task allocation between the 
operator and a computer serving as a backup decision maker when workload was high or the operator was 
unable to perform a task. Like Parasuraman et al. (1996), they used a performance-based approach to the 
adaptive task allocation by continually monitoring human and computer errors rates during task 
performance. The model of human performance developed by Rencken and Durrant- Whyte (1993) was 
used to estimate changes in operator ability to address target classifications in real-time. It was also 
capable of predicting future human and computer performance levels and recommending appropriate task 
allocations. The model was programmed to make a task allocation prediction every time a new target 
arrived in the physical area under surveillance. The computer then decided what new targets the human 
was capable of classifying, given the current task load (i.e., the computer managed the task allocations). If 
the computer determined that operator workload was already too high (based on the human performance 
model), then automation was activated to aid the operator in the new tasks. The system developed by 
Rencken and Durrant- Whyte (1993) functioned in a closed-loop manner in which an explicit task 
allocation between the human and computer was initially determined by the computer, tasks were 
assigned, and performance was measured in real-time and used as a basis for establishing future 
allocations. With respect to the adaptive interface, the computer display was broken-down into modules (a 
graphical image of the room under surveillance, system messaging, system activation controls, target 
labeling controls, display parameter controls) to present different tasks and the behavior of each module 
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was defined separately based on operator workload assessments. The study demonstrated that the use of 
the adaptive interface for task allocation was an effective approach to addressing operator workload and 
ability to perform surveillance tasks in a desired manner. The adaptive interface was implemented in an 
actual surveillance system and compared with manual or unaided human target classification under low 
and high event rates. Human performance and overall system performance were enhanced by the interface 
aid. The system prevented operators from becoming overwhelmed with targets and significantly reduced 
the mean target service time. 

There have been a number of more advanced approaches to adaptive task allocation in complex systems 
presented in the literature, including using physiological variables as a basis for decisions about whether 
and when automation should be invoked. In very recent research, Berka, Levendowski, Cvetinovic, 
Petrovic, Davis, Lumicao, Zivkovic, Popovic and Olmstead (2004) assessed the utility of real-time 
monitoring of operator cognitive states, alertness and memory states using EEG signals as a basis for 
intelligent allocation of tasks between humans and autonomous agents in a complex system. They used a 
warship commander task simulation (a navy command and control simulation) involving multiple, 
simultaneous subtasks requiring the use of various cognitive resources. Subjects were exposed to multiple 
levels of workload in a cognitive task as part of the simulation. They demonstrated that EEG indices 
could be reliably associated with varying levels of cognitive workload and the indexed responses changed 
as a function of subject training and learning and memory requirements as part of task performance. The 
authors said the systems showed good promise for adaptively augmenting cognition in military 
environments. 

Prior to the work by Berka et al. (2004), Pope, Bogart & Bartolome (1995) developed a closed loop 
system to evaluate human-automation interface design, based upon a criterion of mental engagement 
derived from EEG activity. They argued that optimal adaptive task allocation between a human and 
automation is achieved when closed-loop system control is stable, reflecting stable mental engagement of 
the operator. The purpose of this study was to evaluate the usefulness of several different candidate EEG 
indices for reflecting mental engagement. Six subjects performed the MAT-B. The monitoring, 
communication, and resource-management tasks of the battery remained under automated control 
throughout all trials. The tracking task was performed by the subject when in manual mode and monitored 
by the subject when in automated mode. EEG activity was measured at seven scalp sites and total EEG 
power was computed over three frequency bands at each site. Each trial consisted of time spent under 
‘negative’ feedback and time spent under ‘positive’ feedback conditions. Under negative feedback, the 
tracking task was switched to (or remained under) manual control when the engagement index was 
decreasing. Under positive feedback, the tracking task was switched to (or remained under) automated 
control when the engagement index decreased. Each trial was 16 minutes long, with alternating 4-minute 
blocks of positive and negative feedback. Each candidate EEG/mental engagement index was judged on 
the basis of its strength in producing the expected feedback control system phenomena; that is, stable 
operation under negative feedback and unstable operation under positive feedback. The authors 
determined that of all the indices examined, beta/( alpha + theta) reflected task engagement best and was 
most sensitive to changes in task demand. They concluded that their approach represented a dynamic, 
interactive method of adjusting a system design to optimize operator engagement. It is important to note 
here that the automation cycle times observed in this study were even shorter than those observed or 
examined in other adaptive automation research (Hilbum et al., 1993; Scallen et al., 1995) around the 
same period of time. Pope et al. (1995) stated that no condition combination (e.g., manual mode and 
negative feedback) was operative for a continuous interval of more than a few seconds, and they defined 
their run durations for examining EEG activity as the number of 2-second epochs for which the tracking 
task remained in one mode, either manual or automated. 

Although Pope et al. (1995) results showed promise for designing adaptive automation around EEG 
measures, there remained a need for research to demonstrate the utility of physiological-based adaptive 
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aiding for managing complex system operator workload and performance. Prinzel, Freeman, Scerbo, 
Mikulka and Pope (2000) sought to replicate and expand the experiment conducted by Pope et al. (1995). 
They developed a closed-loop system to examine the utility of physiological indicators of cognitive states 
as a basis for driving adaptive task allocation between human operators and automation in a complex 
system and to determine whether such an approach could regulate operator performance and workload. 
They developed a system in which an original EEG-based index of operator arousal was used as a basis 
for determining whether and when automation should be applied to tasks as part of the MAT-B to assist 
operators and moderate workload. Subjects were required to perform either a single task or multiple tasks 
as part of the flight simulation. In the setup by Prinzel et al. (2000), a computer system made task 
allocation decisions based on operator engagement, as indicated by the EEG signal, following either a 
negative or positive feedback control strategy. The study demonstrated that it is possible to manage 
operator workload within a range through a closed-loop system driven by an operator’s own EEG. The 
authors also found the approach to adaptive task allocation to significantly impact operator perceived 
workload measured using the NASA Task Load Index. Participants found the multiple task condition with 
adaptive aiding to be more taxing than the single task condition. 

In a related study, Prinzel, Pope & Freeman (2002) examined the efficacy of psychophysiological self- 
regulation during performance of adaptively automated tasks by assigning groups to either a self- 
regulation condition, false feedback condition, or control condition. Six subjects were assigned to each 
test condition and participants performed the monitoring, resource management and compensatory 
tracking tasks of the MAT-B. Only the tracking task was adaptively automated, and it was done so using 
an EEG engagement index similar to those investigated by Pope et al. (1995) and Prinzel et al. (2000). 
Retum-to-manual-control performance was assessed on the tracking task 3 seconds after a change was 
made in the tracking task mode, and was found to be poorer for subjects in the control and false feedback 
conditions, as compared to subjects in the self-regulation group, who were provided with biofeedback 
regarding their task engagement level. These subjects could use this biofeedback information as a type of 
“cue” to better prepare for and respond to automation state changes compared to those subjects receiving 
no feedback or irrelevant information. These results provide some evidence that advance cueing of 
automation state changes may serve to reduce adaptive automation-induced performance deficits. Further, 
subjects in the self-regulation condition were better able to maintain their task engagement level within a 
narrow range, thereby reducing the need for many task mode changes. The effect of this was an increase 
in task performance as well as a decrease in reported workload. 

Finally, related to the historical research on automation invocation authority in adaptive systems, some 
recent research has provided insight into the effectiveness of computer management of task allocation in 
distributed systems. Galstyan & Leman (2005) developed an approach to the distribution of tasks across 
multiple autonomous agents/robots as part of a single system in order to address environmental demands 
and to promote performance. Such robot systems may operate under human supervisory control, as in 
Parasuraman et al. (2005) research, or they may function autonomously. Galstyan and Lerman (2005) 
designed a computer algorithm to choose among multiple tasks to be performed by multiple robots. The 
robots were then to function collaboratively to achieve the mission of the system. The computer decision 
making regarding the task allocation was based on sensing of local environmental circumstances. The 
results on computer authority over the automation invocation showed improvements in individual robot 
performance and collective robot behavior. The authors said their approach could be used as an algorithm 
or model for adaptive task allocation across multiple autonomous agents. Such models may also apply to 
integrated human-robot systems in which the human acts as another agent working with the robots. 

3.2.1. Summary of Adaptive Task Allocation Research 

Early work in this area provided strong evidence of the potential benefits of adaptive automation for 
supporting human psychomotor performance, even when using simple automation invocation approaches 
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based on task workload models or operator performance histories (e.g., Parasuraman et al., 1996). 
Rencken and Durrant-Whyte’s (1993) research effectively integrated a performance-based approach to 
adaptive task allocation with an adaptive interface technology to accommodate human ability to perform 
complex control system tasks and to moderate operator workload. They showed that a model of human 
performance combined with a computer algorithm for managing dynamic task allocations was useful for 
reducing human error rates and improving overall system performance. This was one of the first adaptive 
task allocation and adaptive interface design investigations. 

More advanced approaches to adaptive task allocation have been explored in the recent literature. The 
study by Pope et al. (1995) was critical in the history of adaptive automation research because it 
demonstrated the potential for EEG signals to be used in indices of operator task engagement and to serve 
as a basis for dynamic task allocations to a human operator and computer. The research by Prinzel et al. 
(2000) was important because it was one of the first studies to reveal the effectiveness of a biocybemetic 
closed-loop system for facilitating adaptive task allocation to regulate workload and promote 
performance. Berka et al. (2004) study also demonstrated that EEG signals may be a useful basis for 
assessing operator cognitive states in real-time in order to drive adaptive task allocations in complex 
automated systems and manage levels of operator engagement. Berka et al. (2004) demonstrated that it is 
possible to use EEG indices for regulating human workload in complex system control under multitasking 
conditions. The findings of the Prinzel et al. (2000) study were extended by the Prinzel et al. (2002), who 
revealed that complex system operators could effectively use biofeedback for self-regulation of task 
engagement states and to better prepare for system control mode changes, as compared to groups 
receiving no feedback. In general, the physiological self-regulation work that has been conducted 
demonstrates how adaptive automation can be used as a practical approach for managing operator 
workload states. Self-regulation approaches may also prevent problems with fast-cycle automation 
(frequent switching among control modes), as observed by Hilburn et al. (1993), and address performance 
loses due to the need for human invocation authority in some systems and increased workload. 

Contemporary work in this area by Galstyan and Lerman (2005) has demonstrated that it is possible to 
adaptively allocate tasks across multiple autonomous agents in a complex system in order to achieve 
higher levels of group behavior. Related to Prinzel et al. (2000) study, Galstyan and Lerman’s (2005) 
computer algorithm or model for managing task allocation proved to be highly effective for performance. 
Such algorithms for multi-agent task allocation may be useful for adaptively prescribing tasks in 
integrated human-robot systems. 

In general, the adaptive task allocation research shows that several approaches to automation invocation 
philosophy and strategies to task allocation are effective for promoting complex, automated system 
performance and for regulating operator workload. The work also lends support to the design of adaptive 
systems in which a computer makes decisions about task allocations over adaptable systems in which the 
human operator is taxed with managing function allocations, in addition to performing mission-related 
tasks. 

3.3. Adaptive Function Allocation Research 

Of the relevant studies identified through the literature search, only two (Scallen & Hancock, 2001; 
Scallen et al., 1995) used the terminology adaptive function allocation in referring to the implementation 
of adaptive automation in complex systems. Adaptive function allocation research has primarily 
examined the performance effects of distribution of low-level system functions, such as controlling one or 
more dimensions of a tracking task, between humans and computers. The studies by Scallen and 
colleagues have specifically focused on determining how pilot performance and workload can be 
improved through dynamic allocation of aircraft functions or tasks based on real-time monitoring of 
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behavior. On a more general level, they have sought to assess the efficacy of adaptive automation for 
multitasking situations in the aviation domain. 

In the study by Scallen et al. (1995), they used a low-fidelity aviation simulation to examine pilot 
responses to rapid cycling automation applied to a flight path tracking task while aircraft systems 
monitoring and fuel management tasks were performed manually in all trials. The adaptive automation in 
this research was implemented using a model-based approach with manual and automated control of the 
tracking task cycling every 15, 30 or 60s. Contrary to the findings of Hilbum et al. (1993), Scallen et al. 
(1995) initially observed significant improvements in tracking task performance with automation cycles 
every 15s as compared to every 60s in the piloting tasks. Performance in the secondary tasks under 
manual control (monitoring and fuel management) was unaffected by the adaptive automation of the 
tracking task. It is important to reiterate that the dynamic function allocations in this study followed a 
predetermined time schedule. The dynamic function allocations in Hilbum et al. (1993) study were truly 
adaptive in nature and based on operator performance levels. In an additional analysis of manual tracking 
performance immediately following the 15s of automation, Scallen et al. (1995) found a different pattern 
of results indicating task performance to be highly sensitive to difficulty manipulations. They concluded 
that excessively short cycles of automation in multitasking situations could prove disruptive to overall 
performance. 

Hadley, Prinzel, Freeman and Mikulka (1999) conducted a study to further investigate the affects of short 
cycle automation on operator performance and workload in complex systems control. In general, they 
found that reversions to manual control after short periods of automation had a negative effect on 
subsequent performance and appropriate cognitive resource allocation to tasks. They said there might be 
greater manual performance problems associated with shorter cycle automation. They observed that 
participants found it more difficult to reorient to, and sustain, manual control of a compensatory tracking 
task as the duration of a preceding automated control period decreased. (A P300 evoked response 
potential was significantly smaller for shorter-cycles indicating reduced perception of the mode transition 
and new information processing needs.) There was also a significant increase in workload for shorter 
cycle automation. However, this study and the research of Scallen et al. (1995) only addressed the 
problem of retum-to-manual performance problems for automation applied to psychomotor functions as 
part of complex systems control and not cognitive functions. 

In general, the studies of adaptive automation examining the influence of automation cycle time on 
operator performance have revealed longer periods of automated control, offset with manual control, to 
better support performance, as compared to short cycle automation. Long duration automation cycles also 
appear to facilitate workload reductions. 

In more contemporary research, Scallen & Hancock (2001) sought to provide additional information on 
the effects of adaptive function allocation on pilot performance and workload by manipulating the 
approach to automation invocation in tasks and the underlying function allocation strategy. In the 
STARFIRE flight simulation, automation of the tracking task occurred at the time of critical events 
(surface targets appearing) and on the basis of operator performance, and the automation was applied to 
either both dimensions of 2-D tracking or one dimension (vertical or horizontal). Pilots were also 
required to manually perform aircraft systems monitoring and fuel management in all trials. The adaptive 
automation conditions evaluated in the experiment all appeared to be effective for sustaining tracking 
performance compared to manual control, even under high workload. Furthermore, adaptive automation 
of the tracking task appeared to benefit secondary monitoring and fuel management task performance. 
Interestingly, the effect of the semi-automated tracking task allocations was equivalent to the fully 
automated allocation; therefore, a complete shift in tracking control from human to automation was not 
necessary to achieve performance benefits. However, Scallen and Hancock (2001) did note increased 
variability in pilot performance with partial automation of the tracking task, as compared to the other 
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conditions. The authors suggested that pilots could be retained in the control loop by using a part-task 
allocation strategy to maintain high performance in the cockpit, without compromising manual skill over 
the long term. The authors also advocated the use of adaptive automation for a wide range of pilot 
functions, such as task prioritization, mission segmenting, task initiation and cessation, risk identification, 
and workload management. 

This research was important because it demonstrated that adaptive automation could be effectively 
applied to specific low-level functions of a task versus automating an entire task in order to achieve 
performance benefits. It is possible that adaptive automation of low-level functions (instead of tasks) may 
provide greater flexibility for managing operator workload states and, at the same time, support the 
achievement of situation awareness. 

4. Adaptive Interface Technology 

Of the search terminology used for this review, adaptive interface(s) generated the largest number of 
references, approximately 42; however, only 16 of the studies proved to have relevance to the design of 
adaptive automation for complex systems. Furthermore, some of the studies considered relevant to this 
research were found to target desktop computing applications versus complex interfaces used to control 
machine systems in dynamic real-world environments. 

The terminology adaptive automation has historically been used to refer to dynamic allocation of control 
between human and computer servers in real-world systems, based on predefined function allocation 
schemes and various system-state variables. The adaptation of automation implies changes in machine 
behavior, including functional responses to operator commands and the type of feedback provided by a 
system to the user. It also implies changes in the task responsibility of operators ranging from monitoring 
to control action implementation. Adaptive automation may be considered in the development of software 
design when changes in application states amount to more than simple transitions from one interface 
display configuration to another and include activation or deactivation of specific software subroutines for 
handling data input from users, facilitating information processing, and presenting output. 

Adaptive interface technologies may be used to support, or provide access to, adaptive automation 
programmed in software applications. Different instances of a display interface and controls may allow 
users to exploit specific software functions available under different modes of automation. In this way, 
the adaptive interface technology is critical for presenting and promoting the effectiveness of the adaptive 
automation, as in the early research by Rencken and Durrant- Whyte (1993). However, in some cases, the 
design of an adaptive interface does not necessarily imply adaptation of underlying system functionality 
to user needs. 

Research on adaptive interface technologies for supporting human-machine system performance appeared 
in the engineering literature as early as the 1980s (e.g.. Innocent, 1982; Mason & Thomas, 1984; Hancock 
& Chignell, 1988), slightly more than five years after the development of the personal computer. These 
studies focused on developing self-adapting interfaces for man-machine systems, mental workload 
responses associated with the use of adaptive interface technology, and potential applications of adaptive 
interfaces in information systems. Additional studies occurred in the 1990s and focused on developing 
new approaches to interface design for software applications to create intelligent and adaptive 
technologies (Sukaviriya, 1993). The basis for these technologies was to create descriptive user models by 
observing task behaviors with applications. The models were integrated with interface adaptation 
strategies specifying detailed changes in interface features. One of the main challenges identified through 
this research was the need to relate specific instances of a user interface to certain user states. Sukaviriya 


14 



(1993) worked to identify high-level li nks between aspects of user tasks and the activation of interface 
components/features (e.g., how to sequence dialogs, how to generate help information). 

4.1. User Modeling as a Basis for Adaptive Interface Design 

Several subsequent studies (Debevc, Meyer, Donlagic & Svecko, 1996; Gong & Salvendy, 1995; Arai, 
Fukuda, Yamamoto, Naito & Matsui, 1996; Yoshida & Motoda, 1996) identified the need to consider 
user needs for specific applications, individual differences in the ability to use software, personal 
differences and preferences, and individual behaviors in designing user-adapted interfaces. Therefore, one 
major thrust of this research area has focused on individual user modeling as basis for implementation of 
adaptive interfaces. Debevc et al. (1996) explored how to adjust computer display features and software 
use procedures to support individual patterns of work. They assessed adaptive icon toolbars that allowed 
for addition or removal of icons in real-time (controlling function access) based on a model of user usage 
history and task needs, for supporting desktop computing tasks. An important aspect of this technology 
was that users had to agree to or veto adaptive interface features as they became available during task 
performance. The authors observed that in the use of the adaptive toolbar there was a need for certain 
aspects of the overall software interface to remain consistent to allow users to apply a general model of 
the system for performance. The adaptive interface was found to effectively support user performance. 

Gong and Salvendy’ s (1995) research was similar in that they developed adaptive menu and command 
interfaces that could be modified based on changing levels of user skill due to task learning. They 
conducted an experiment with subjects revealing performance to be significantly improved with the 
adaptive interface versus static menus or command interfaces. Interestingly, there was no impact of the 
adaptive interface technology on perceived memory load or satisfaction ratings relative to the 
conventional, static interfaces. 

Arai et al. (1996) took a more complex approach to adaptive interface design considering the interactive 
relationship between a user and computer and how each adapts to the other. Factors they considered in 
modeling this interaction included characteristics of the task, the state of the user (individual 
characteristics, skill level and behaviors), and physical/environment conditions. They developed an 
adaptive virtual reality (VR) simulation interface that changed states based on user performance of the 
VR game and galvanic skin response, as an indicator of arousal level. They presented results 
demonstrating that the VR interface adaptation accommodated different user states for performance 
improvements over a static interface. 

Yoshida and Motoda (1996) conducted a study with a premise similar to that of the Debevc et al. (1996) 
work. They emphasized the need for elaborate user models for adaptive interface development. The main 
problem identified by Yoshida and Motoda (1996) was that prior research had created models largely 
based on activity/command sequences and behaviors at interfaces. They said that such an approach was 
inadequate for predicting user behavior under different task circumstances. They presented an approach to 
user modeling that not only considered command sequences but also the aspects of cognition activated as 
part of the sequences in order to create more elaborate models. They used a simple desktop computing 
application (the ClipBoard) to present an adaptive interface and to demonstrate the adequacy of their 
model for predicting user performance and as a basis for adaptive interface design. 

In the last study on adaptive interfaces (identified by the literature search) to occur in the 1 990s, Miller 
(1999) discussed the limitations of existing approaches to adaptive interface design and recommended a 
new method to ensure the “fit” of interfaces to operator information demands. He said that prior work had 
offered simple approaches based on pattern matching rules associating a few instances of an interface 
with a limited set of parameter settings defining user needs. His approach was to formally identify 
information requirements of user tasks and to associate these requirements with the capability of different 
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interface features for conveying information. The interface design approach was demonstrated through 
prototyping of an adaptive interface for military attack helicopters. 

4.2. Performance with Adaptive Interfaces and Extensions of User Models 

With the new century, several investigations were conducted to assess the utility of adaptive interfaces for 
supporting human performance in complex systems control, including adaptive displays and controls for 
Air Force pilot performance (e.g., Haas, Nelson, Reppberger, Bolia & Zacharias, 2001; Bennett, Cress, 
Hettinger, Stautberg & Haas, 2001). In the Bennett et al. (2001) study, the authors initially presented 
theoretical background on dynamic adaptive interfaces (DAIs) (interfaces involving changes in both 
displays and control of a system in real-time) and they compared them to forms of automation and 
decision aiding. They said that the goal of such interfaces is to anticipate the needs of an operator and to 
provide necessary information without requiring explicit control actions. This is different from the 
adaptive interface design approach presented by Debevc et al. (1996) for desktop computing applications 
where users were required to accept or veto changes in an adaptive interface in real-time. Bennett et al. 
(2001) laid-out a conceptual framework for design and evaluation of DAIs to explore specific design 
issues. They said that if adaptive interfaces are designed poorly, they have the potential to increase 
workload and degrade human performance. They identified different types of information that could serve 
as bases for triggering adaptive automation/interface changes for decision support in aviation tasks, 
including variability in the task environment (significant events), data on the human operator states (real- 
time assessments of workload and performance), and on-line assessment of operator activities (the tasks 
being performed and operator intent). They used a low-fidelity (F-16) flight navigation simulation to test 
experienced pilots in a waypoint following task in short trials. They developed and tested three types of 
interfaces including: a standard interface with basic throttle and joystick controls and a standard heads-up 
display (HUD); a candidate interface with a force-reflecting joystick and configural HUD; and an 
adaptive interface incoiporating aspects of both of the former conditions (dynamically). Under the 
adaptive condition, pilots could alternate between use of the standard and candidate interfaces. Bennett et 
al. (2001) also manipulated workload through simulated turbulence conditions. They observed subject 
performance measures, including aircraft position relative to the pre-planned flight path, and they used 
deviation from the path as a basis for adaptation of the interface content. They found the workload 
manipulation to significantly effect performance and the candidate and adaptive interfaces to improve 
performance quality relative to the standard condition. However, they did not find the adaptation 
capability of the interface to promote performance beyond that achievable with the candidate interface 
alone. Furthermore, subjects generally found the adaptive interface to produce higher workload. 

Other research at this same time returned to the need to make elaborate consideration of user 
characteristics in developing adaptive interface technologies (Duvallet, Boukachour & Cardon, 2000) and 
some studies also focused on Air Force applications (Hudlicka & McNeese, 2002). The research by 
Duvallet et al. (2000) was focused on developing adaptive interfaces for information systems to support 
decision making. They said that for an interface to be adapted to individuals, many characteristics need to 
be considered, including ways of perceiving information, patterns of interface behavior, and how 
environmental factors affect behavior. Duvallet et al. (2000) said there remains a need for user models to 
take into account these factors for adaptive interface design. This line of thinking is very similar to the 
elaborate user modeling approaches advocated by Arai et al. (1996) and Yoshida and Motoda (1996). 
Duvallet et al. (2000) also said that with respect to implementing adaptive interface technologies, addition 
or subtraction of components/features should occur without making the overall design of the interface 
uncertain for operators. This notion is similar to the observation by Debevc et al. (1996) that there is a 
need to maintain some consistent features in an adaptive interface from one mode to another in order to 
allow users to apply a general system model for performance. 
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Similar to the Bennett et al. (2001) research, Hudlicka and McNeese (2002) presented a framework for 
adaptive interface design incorporating different user factors, including emotional states and active 
beliefs. They provided an approach for assessing user states, including knowledge elicitation, self-reports 
of stress states, performance on diagnostic tasks and physiological sensing, and relating states to user 
interface adaptation strategies varying based on content and format. They said that the implementation of 
adaptive interfaces to accommodate user needs involves sensing and inferring states and beliefs about a 
system, identifying the potential impact of user states on system performance, selecting an adaptation 
strategy for the user state, and implementing the strategy through specific interface features. In general, 
the approach takes into account user emotional states and the knowledge requirements of a task for 
interface adaptation. Hudlicka and McNeese (2002) also demonstrated the use of their framework for 
prototyping an interface for use in an Air Force combat task simulation. The interface was able to adapt to 
different pilot anxiety levels in the simulation and belief states by modifying specific cockpit instrument 
displays in response to detected pilot state changes. 

In the most recent research on adaptive interface technology, studies have focused on design for Internet- 
based tasks (Ji & Salvendy, 2002; Narayanan, Koppaka, Edala, Loritz & Daley, 2004). Narayanan et al. 
(2004) were concerned with developing approaches to define how displays and available controls in an 
interface could be automatically adjusted to the current goals and abilities of a user by assessing (in real- 
time) user status, the system task, and working context. This work is similar to the historical approach to 
adaptive interface design advocated by Duvallet et al. using elaborate user models. Narayanan et al. 
(2004) said that with adaptive interfaces not only the format but the content must be adaptable for 
supporting user knowledge acquisition tasks. They offered that the requirements for designing adaptive 
interfaces to support these tasks can be identified through knowledge elicitation with users and looking at 
patterns of user behavior. (It is important to recall here that Yoshida and Motoda (1996) said that simply 
inspecting patterns of user behavior is not sufficient for developing complex user models to drive 
adaptive interface design.) Narayanan et al. (2004) said that observing patterns of user behavior is 
particularly useful for developing models of how people search for information and accomplish 
knowledge discovery. They suggested that another approach to modeling users is to analyze the type and 
content of feedback they are able to elicit from applications and to use this as a further basis for 
establishing a user profile and information needs. The profile can be used as a basis for appropriately 
structuring content for users during information search tasks. Obviously, the results of this research are 
most applicable to database application or Internet search engine use. There could be extensions of this 
work for designing adaptive interfaces for real-world automated systems that may require human 
operators to perform information acquisition tasks collaboratively with automation aids. 

4.3. Summary of Adaptive Interface Research 

Most research on adaptive interfaces has been conducted in the context of desktop computing applications 
concentrating on flexible menu design, toolbar design, etc. (e.g., Debevc et al., 1996; Gong & Salvendy, 
1995). One major limitation of this work, relative to design of adaptively automated systems, is that 
desktop applications are typically self-paced, unlike real systems in which operators may not have 
complete control over the pace of interaction with the automation. Early empirical evidence with desktop 
systems revealed that simple adaptive interface technologies may provide performance benefits over 
conventional, static interfaces, even when humans need to make decisions about changes in interface 
features in real-time. Of course, there may be an increase in task workload with this type of interface that 
should be considered on top of the cognitive load imposed by mission-related tasks, and any potential for 
performance decrements should be identified. According to Miller (1999), good adaptive interface design 
starts with the identification of user goals in a task and user information requirements for achieving goals. 
Any instance of an adaptive interface should be evaluated against the user’s goal set and objective 
approaches should be used to quantify the “fit” of an interface to user information requirements during 
task performance. Related to Miller’s (1999) research, the study by Hudlicka and McNeese (2002) 
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provided evidence that there may be a need to consider user emotional states for effective adaptive 
interface design in addition to goal states and information requirements for task performance. Related 
challenges include the quantification of emotional states and the development interface features that 
address specific states, like anxiety. 

Beyond user goals and emotions, many historical studies have advocated elaborate user models as a basis 
for designing adaptive interfaces. The majority of research on adaptive interface technology has focused 
on consideration of user behavior patterns (sequences of interface actions) and preferences as bases for 
design. In constructing adaptive interfaces, Yoshida and Motoda (1996) said that elaborate user models 
are needed that describe the aspects of information processing required for performance of different tasks. 
This is relevant to the design of interfaces for adaptive automation applied to various stages of 
information processing in complex systems control, as discussed above. The models must be capable of 
predicting what the user’s behavior will be with the system interface during information acquisition, 
information analysis, etc. Interfaces should then be designed based on the model in order to best 
accommodate specific user behaviors. Duvallet et al. (2000) reinforced this point by saying that many 
user characteristics need to be incoiporated in models as a basis for adaptive interface design. The 
Narayanan et al. (2004) work also emphasized the need for elaborate user models based on real-time 
assessments of user status, system or task states, and the environmental context for adapting interfaces to 
user needs. Beyond looking at patterns of detailed user behavior with interfaces, examining the type of 
output being generated by systems for users may provide another basis for profiling users for future use in 
interface adaptation strategies (Narayanan et al., 2004). 

With respect to real-time assessment of user states for triggering adaptive interface changes, the research 
by Arai et al. (1996) may have been one of the first investigations to use physiological response measures 
for real-time characterization of operator stress states and use in driving adaptive interface settings. This 
is akin to the adaptive task allocation research using physiological variables for indicating cognitive 
workload states and prescribing appropriate allocations of tasks to a human or computer in complex 
systems control with the objective of promoting overall performance (e.g., Prinzel et al., 2000; Wilson & 
Russell, 2003). In addition to adapting interfaces to operator stress or workload levels, Gong and 
Salvendy (1995) observed that as users become more familiar with systems over time, there may be an 
additional need for adaptive interfaces to be designed to accommodate skill changes. This is particularly 
relevant to adaptive automation systems for which learning curves may be steep. 

With respect to implementing interfaces changes to accommodate operator states, Duvallet et al. (2000) 
offered that the addition or subtraction of features in dynamic adaptive interfaces should not lead to user 
uncertainty about the overall design of the interface. Some elements should remain consistent from mode 
to mode as references for operators. Prior to this Debevc et al. (1996) said there is a need for some 
consistent features in adaptive interfaces, as an operator transitions from one state of the interface to 
another, in order for them to apply some general task model for use of the system. 

In regard to supporting human performance with adaptive interface technologies in complex systems 
control, Bennett et al. (2001) demonstrated that there may be new advanced interface technologies for 
supporting operator performance (e.g., contemporary HUDs for fighter aircraft cockpits) that are superior 
to conventional interfaces and can be adaptively applied based on task circumstances. However, the 
reasons for making the technology adaptable and the manner in which the adaptation occurs is critical to 
promoting performance over static use of new technologies. In designing adaptive interfaces to, for 
example, provide pilots with the flexibility to shift between standard controls and advanced controls, it is 
important to ensure that the switching capability has meaning with respect to the flight tasks to be 
performed. Otherwise, the adaptation may not yield benefits beyond advanced control use, alone. Each 
mode of an adaptive interface should have some benefit for specific aspects of a task. 
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Related to Bennett et al. (2001) research, Bubb-Lewis and Scerbo (2002) conducted an empirical 
investigation demonstrating that adaptive interface technologies can provide flexibility in human- 
computer interaction that may promote performance over conventional static interfaces. They said that 
restrictions in human-computer communication due to traditional interface design can lead to 
performance decrements. However, they also observed that in adaptive systems, when automation 
assumes greater control of a task, there are increased restrictions on communication between the human 
and automation and performance on simple tasks may decline. This is another negative implication of 
adaptive interface technology that should be considered in design. 


5. Discussion 

One of the main findings of this review is that adaptable system designs requiring human delegation of 
task and function authority to automation during performance may pose additional workload on operators 
leading to degradations in situation awareness and performance, as compared to adaptive automation. 
Based on the review of literature, current approaches to adaptable system design (e.g., the playbook 
approach) may only be effective for addressing finite operating states of systems or facilitating rule-based 
operator behaviors. However, such automation may ultimately only produce performance comparable to 
manual control under variable environmental conditions. Furthermore, when workload is high and 
environmental dynamics occur, operators may revert to manual control for effecting knowledge-based 
behaviors. The one advantage that has emerged in research on adaptable automation is that the number of 
interface actions required to performance certain tasks with complex systems may be significantly 
reduced, as compared to manual control. One would expect this to translate into reductions in perceived 
workload but there is currently no evidence to support this. Beyond this, when system operators are 
required to switch between control strategies or tasks on a regular basis, adaptable automation appears to 
provide some advantage over static automation but it is not superior to manual control. 

In contrast to adaptable systems, adaptively automated systems provide for dynamic task/function 
allocations mandated by a computer based on real-time monitoring of operator workload states. Decision 
making regarding the allocation of system control is not an additional responsibility of operators and, 
consequently, overall workload may be lower. Based on the review of literature, numerous approaches to 
adaptive automation (model-based, performance-based, workload-based, etc.) have been shown to 
improve operator performance in both psychomotor and cognitive tasks and to facilitate workload 
reductions in comparison to completely manual control and static automation. These findings are far more 
promising than the current results on adaptable systems performance. Studies of automation invocation 
authority in adaptive systems have also revealed an advantage of computer mandates of function 
allocation over human delegation of tasks to a computer because of potential workload issues. Shared 
approaches to function allocation in adaptive systems may also be appropriate depending on the control 
mode under which the human makes decisions about task delegation. 

Research on adaptive task allocation has shown that sophisticated approaches to real-time assessment of 
operator states (e.g., EEG signals) can be useful for implementing adaptive automation and for promoting 
operator performance in single and multitasking scenarios. Biofeedback also appears to be a useful basis 
for physiological self-regulation in adaptively automated systems control. The research in this area also 
supports the use of multiple automation invocation philosophies and function allocation strategies for 
managing operator workload. Results also support computer management of adaptive task allocation in 
complex systems. 

Research on adaptive function allocation has extended some of the findings on adaptive task allocation by 
demonstrating that adaptive automation can be applied to specific low-level functions as part of larger 
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information processing tasks in complex systems control. Research in this area has also provided insight 
into appropriate durations of automated and manual control as part of adaptive automation for supporting 
performance. 

Research on adaptive interfaces represented the largest body of work covered by this review. Adaptive 
interface technology is important to the implementation of adaptive automation in terms of providing 
operators with displays and controls for access to relevant system functions during different tasks or 
modes of operation. Several studies have identified users characteristics that need to be taken into 
consideration in the design adaptive interfaces. The list includes goal states, information requirements, 
emotional states, skill level, perceptual preferences, interface behavior patterns, information processing 
requirements associated with tasks posed to an adaptive system, etc. The general approach to adaptive 
interface design presented in the literature involves creating elaborate user models and using them as a 
basis for prototyping modes of displays and controls that can address user needs under various task and 
environmental circumstances. For DAIs (dynamic adaptive interfaces), these considerations need to be 
combined with real-time assessment of operator states and the use of galvanic skin response for 
identifying stress states has been demonstrated as an effective measure for adaptive interface changes. 

One major recommendation that can be made based on the adaptive interface research is that some 
features of interfaces should remain consistent across modes of system operation to allow operators to 
rely on, or apply, general mental models of a task or system in using the interface. This is in-line with 
established basic principles of screen design for human-computer interaction tasks (Tullis, 1997). The 
other major recommendation is that when it is possible to provide complex system operators with 
adaptive interfaces, one should ensure that each mode of an interface has relevance to a certain task or 
functional requirement. It is also important for operators to understand which interfaces to use when (and 
why) from performance and workload perspectives. 


6. Conclusions and Future Research 

In general, the findings of this review motivate further exploration of adaptively automated systems for 
managing operator workload states and attempting to promote situation awareness, as compared to design 
of adaptable system alternatives. There is a substantial body of research on adaptive automation 
demonstrating performance and workload benefits over manual systems control and traditional, 
technology-centered approaches to automation. Unfortunately, the same cannot be said for adaptable 
systems and there is mounting evidence of negative performance, workload and situation awareness 
implications associated with requiring human operators to manage dynamic task or function allocations in 
complex systems control. Further research is needed in this area, however, to evaluate different 
approaches to adaptable system design, extending beyond those tested in recent research, and to confirm 
performance decrements of adaptable automation relative to adaptive automation across a range of 
applications. 

The main adaptive automation research directions that need to be explored in the future include: 

(1) Identification of types and levels of adaptive automation that are easiest for operators to work with in 
different task contexts - In general, there is a need for additional investigations on the differential 
effectiveness of adaptive automation applied to various information processing functions in complex 
systems control. Studies need to be conducted on a broad range of applications and should make use 
of high-fidelity simulations when field implementations are not possible. 


20 



(2) Development of more integrated and robust approaches to operator and system state monitoring for 
triggering task/function allocations - There is a need for evaluation of new approaches to automation 
invocation in adaptive systems considering multiple operator and system-state variables 
(simultaneously) as a basis for triggering dynamic task or function allocations. For example, 
combination of performance and workload-based approaches to adaptive automation should be 
considered for making more accurate function assignments in real-time in order to manage operator 
workload and promote performance. Beyond this, EEG-based approaches for assessing operator 
workload states must be refined to rely on a small set of signals for computing indices of arousal that 
can be related to automation states. 

(3) Development of highly accurate methods for operator state classification and computer triggering of 
adaptive task/function allocations - Beyond identifying operator and system variables that may be 
useful for characterizing workload and performance conditions in real-time, there is a need to develop 
advanced computational method to integrate data on large sets of variables for effectively classifying 
system states (e.g., “underloaded”, “overloaded”). Neural network-based approaches have historically 
proved useful for this purpose (Wilson & Russell, 2003); however, they provide little basis for 
explanation of the relationship between, for example, operator physiological responses (ECG, EEG, 
and EOG data) and objective measures of task load. 

Related to this, one of the most important research needs in the adaptive automation area is to 
establish objective human performance and workload criteria for making decisions about whether and 
when to turn “on” or “off’ automation of a particular task or function. Previous investigations have 
used preliminary experiment data or statistical measures for establishing workload criteria to identify 
operator “underload” or “overload” states (e.g., Kaber & Riley; Kaber et al., 2005). New criteria 
expressed in terms of information processing limitations would be far more useful across applications 
and systems (e.g., visual overload (Mackworth, 1976); working memory limitations (Miller, 1956)). 
Another approach would be to develop computational cognitive models of users (e.g., GOMSL 
models, ACT-R models) for use in predicting potential information requirements and automation 
assistance needs in real time. These predictions could be used as a basis for dynamic function 
allocations managed by a computer. 

(4) Development of on-line methods for assessing adaptive automation effectiveness for operator 
workload management and maintenance of situation awareness - There remains a need to establish 
the efficacy of adaptive automation in complex systems and the capability for such human-centered 
approaches to automation to control operator workload and to support achievement of situation 
awareness. Recent research on the performance, workload and situation awareness implications of 
adaptive automation (Kaber & Endsley, 2004) has demonstrated that dynamic allocations of lower- 
order information processing functions (e.g., information acquisition, information analysis) between 
humans and machines may have negative effects on operator situation awareness (over the long- 
term), as compared to short-term benefits for performance and workload. This is an issue that needs 
further empirical investigation and should be considered carefully by adaptive system designer. 
Developing methods for collecting real-time speed and accuracy data on operators during complex 
systems control will allow for assessment of the magnitude and timing of effects of adaptive 
automation on operator behavior and overall system performance. 
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